Currently, one of the common means for interfacing with a small computer such as a PC comprises a screen, a keyboard, and a pointing device (e.g., a mouse). The pointing device arrangement includes an icon that is presented on the screen, a moveable "mouse" connected to the computer, software for moving the icon in response to the movement of the "mouse" and software for recognizing the position of the icon.
The icon mouse arrangement is very effective for obtaining information from the user when the amount of information that is needed is small, such as when computer commands are, effectively, selected. At such a command selection session, the computer presents various options to the user and the user selects from among the presented options.
The schema for presenting the options and for identifying the selection made by the user, via the icon mouse interface, involves creating an image on the screen where specified areas on the screen correspond to specific selections, identifying the position of the icon on the screen, and establishing a correspondence between the icon position, the established screen areas, and the effective command selection.
Typically there is no logical relationship between one area of a screen and another area of the screen. That is, no action that is associated with one area is taken simply because the icon is in some proximal relationship to that area. The question is not asked whether the icon is to the right, left, above, or below area, but only whether the icon is in the area. That is not to say that logical relationships cannot exist between areas, and in some applications they do. However, the programs that recognize and implement the controls associated with such logical relationships are "tailor made" for the specific applications.
Aside from the "control" aspect of the interaction between the computer and the user, there are other situations where different areas of a display screen share a logical relationship. For example, when an image of a business letter (of a known format) is formed on the screen, it is known where the date of the letter can be found, where the addressee can be found in relation to the date, where the salutation can be found in relation to the addressee, etc. The same applies to many database output "screens", which often are presented in tabular form.
The question is: how to program the computer to recognize and analyze these logical relationships, and how to approach the programming task in a logically robust way. A careful study of this question reveals that it basically is a grammar problem. That is, the problem involves the question of what is the best way to parse the image and what is the best way to make sense of the parsed information.
Parsing of information and deriving contextual information as a result of the parsing has been studied extensively in connection with formal languages. Such languages comprise a collection of symbols and a set of rule-defined manipulations of the symbols (grammar). The set of symbols includes both terminal symbols, which are the elemental units in the language (such as words) and non-terminal symbols which are constructed from terminal symbols (such as phrases). The symbol manipulation rules are sometimes termed the production rules, and those define valid mappings of symbols. For example, the production rule EQU A.fwdarw.bcN (1)
states that when the sequence of symbols bcN is found in the input, that sequence may be replaced with the symbol A. (In this disclosure, lower case letters designate terminal symbols and upper letters designate non-terminal symbols.) Obviously, the notion of "next" and "previous" is very important to the production rules, and that imposes a strict concept of sequencing on the symbols in any candidate "sentence" of the language. The ordering of symbols is clearly definable in one dimensional strings (which natural languages are). In higher order strings, however, the concept of a sequential order is not well defined. That is the reason why parsing of multi-dimensional strings has not progressed very far.